Complexity L0-Penalized M-Estimation: Consistency in More Dimensions

Complexity L0-Penalized M-Estimation: Consistency in More Dimensions

We study the asymptotics in L for complexity penalized least squares regression for the discrete approximation of finite-dimensional signals on continuous domains—e.g., images—by piecewise smooth functions. We introduce a fairly general setting, which comprises most of the presently popular partitions of signal or image domains, like interval, wedgelet or related partitions, as well as Delaunay...

Multiscale Likelihood Analysis and Complexity Penalized Estimation

We describe here a framework for a certain class of multiscale likelihood factorizations wherein, in analogy to a wavelet decomposition of an L function, a given likelihood function has an alternative representation as a product of conditional densities reflecting information in both the data and the parameter vector localized in position and scale. The framework is developed as a set of suffic...

Weak convergence of the regularization path in penalized M-estimation

We consider an estimator b βn(t) defined as the element φ ∈ Φ minimizing a contrast process Λn(φ, t) for each t. We give some general results for deriving the weak convergence of √ n(b βn − β) in the space of bounded functions, where, for each t, β(t) is the φ ∈ Φ minimizing the limit of Λn(φ, t) as n → ∞. These results are applied in the context of penalized M-estimation, that is, when Λn(φ, t...

M-estimation and complexity regularization

Lectures 12 and 13 - Complexity Penalized Maximum Likelihood Estimation

As you learned in previous courses, if we have a statistical model we can often estimate unknown “parameters” by the maximum likelihood principle. Suppose we have independent, but not necessarily identically distributed, data. Namely, we model the data {Yi}i=1 as independent random variables with densities (with respect to a common dominating measure) given by pi(·; θ), where θ is an unknown “p...